Pulse signal circuit, parallel processing circuit, pattern recognition system, and image input system

ABSTRACT

A synaptic connection element for connecting neuron elements inputs a plurality of pulsed signals from different neuron elements N 1  through N 4,  effects a common modulation (time window integration or pulse phase/width modulation) on a plurality of predetermined signals among the plurality of pulse signals, and outputs the modulated pulse signals to different signal lines to a neuron element M 1.  A neural network for representing and processing pattern information by the pulse modulation is thereby downsized in scale.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a pulse signal circuit and further to aparallel processing circuit and a pattern recognition system that usethis pulse signal circuit, and an image input system for detecting aspecified object etc. by use of the pattern recognition system.

2. Related Background Art

Image and voice recognition implementation systems have hitherto beenroughly classified into such a type that a recognition processingalgorithm specialized for a specified recognition object is sequentiallyoperated and executed as computer software, and a type in which the samealgorithm is executed by a dedicated parallel image processor (such asan SIMD (Single Instruction Multiple Data) processor, an MIMD (MultipleInstruction stream/Multiple Data stream) processor and so on).

Typical examples are given below as exemplifying the image recognitionalgorithm. At first, the following is methods involving calculating afeature amount relative to a similarity to a recognition object model.One method is a method for representing recognition object model data asa template model, calculating a similarity by template matching etc withan input image (or a feature vector thereof) and calculating ahigh-order correlation coefficient. Another method is a method(Sirovich, et al., 1987, Low-dimensional procedure for thecharacterization of human faces, J. Opt. Soc. Am. [A], vol. 3, pp.519–524) for mapping an input pattern to an intrinsic image functionspace obtained by analyzing primary components of an object model image,and calculating an intra-feature-space distance from the model. Afurther method is a method (Lades et al., 1993, Distortion InvariantObject Recognition in the Dynamic Link Architecture, IEEE Trans. onComputers, vol. 42, pp. 300–311) for representing a plurality of featureextraction results (feature vectors) and a spatial arrangementrelationship as graphs, and calculating a similarity based on elasticgraph matching. A still further method is a method (Seibert, et al.,1992, Learning and recognizing 3D objects from multiple views in aneural system, in Neural Networks for Perception, vol. 1 Human andMachine Perception (H. Wechsler Ed.) Academic Press, pp. 427–444) forobtaining position-, rotation- and scale-invariable representations byexecuting predetermined conversions with respect to input images andthereafter collating with a model.

The following is exemplifications of a pattern recognition method basedon a neural network model of which a hint is acquired from a biologicalinformation processing system. One exemplification is a method (JapanesePatent Post-Exam. No. 60-712, Fukushima and Miyake, 1982, Neocognitron:A new algorithm for pattern recognition tolerant of deformation andshifts in position, Pattern Recognition, vol. 15, pp—455–469) forimplementing hierarchical template matching. Another exemplification isa method (Anderson, et al., 1995, Routing Networks in Visual Cortex, inHandbook of Brain Theory and Neural Networks (M. Arbib, Ed.), MIT Press,pp. 823–826) for obtaining object-based scale- and position-invariablerepresentations by dynamic routing neural networks. Otherexemplifications are methods using multi-layer perceptrons, a radialbasis function network and so on.

On the other hand, what is proposed as a scheme for taking aninformation processing system based on biological neural networks with ahigher fidelity, is a neural network model circuit (Murray et al., 1991,Pulse-Stream VLSI Neural Networks Mixing analog and digital Techniques,IEEE Trans. on Neural Networks, vol. 1.2, pp. 193–204; Japanese PatentApplication Laid-Open Nos. 7-262157, 7-334478 and 8-153148, and JapanesePatent Publication No. 2879670) for transmitting and representinginformation through on a pulse train corresponding to an actionpotential.

Methods for recognizing and detecting a specified object by a neuralnetwork constructed of pulse train generation neurons, are systems (U.S.Pat. No. 5,664,065, and Broussard, et al., 1999, PhysiologicallyMotivated Image Fusion for Object Detection using a Pulse Coupled NeuralNetwork, IEEE Trans. on Neural Networks, vol. 10, pp. 554–563, and soforth) using a pulse coupled neural network (which will hereinafter beabbreviated to PCNN), to be specific, a high-order (second-order orhigher) model by Echhorn (Eckhorn, et al., 1990, Feature linking viasynchronization among distributed assembles: simulation of results fromcat cortex, Neural Computation, vol. 2, pp. 293–307) which is based onthe premise of linking inputs and feeding inputs.

Further, a method for relieving a wiring problem in the neural networkis an event-driven oriented method (Address Event Representation: thiswill hereinafter be abbreviated to AER) (Lazzaro, et al., 1993, siliconAuditory Processors as Computer Peripherals, In Touretzky, D (ed),Advances in Neural Information Processing Systems 5. San Mateo, Calif.:Morgan Kaufmann Publishers) for coding addresses of so-called pulseoutput neurons. In this case, IDs of pulse train output-sided neuronsare coded as binary addresses, whereby even when output signals from thedifferent neurons are arranged in time sequence on a common bus, theinput-sided neurons are able to automatically decode the addresses ofthe source neurons.

On the other hand, the neural network processor related to JapanesePatent Publication No. 2741793 schemes to reduce the number of neuronsand to downsize a circuit by configuring multi-layered feedforwardoriented networks in a systolic array architecture.

Each of the prior arts described above, however, still entails, asproblems to a great extent, a difficulty of downsizing both wiringportions related to inter-neuron connections and a circuit scale ofsynaptic connection circuit of which the number is by far larger thanthe number of neurons, and a difficulty of applying a layout ofrespective components to a general pattern recognition.

SUMMARY OF THE INVENTION

It is therefore a primary object of the present invention to actualizean equal performance in a smaller circuit scale than in the prior artsby sharing a synaptic connection circuit.

According to one aspect, the present invention which achieves theseobjectives relates to a pulse signal processing circuit comprising amodulation circuit for inputting a plurality of pulsed signals fromdifferent arithmetic elements and modulating in common a plurality ofpredetermined signals among the plurality of pulse signals, wherein themodulated pulse signals are outputted in branch to different signallines, respectively.

According to another aspect, the present invention which achieves theseobjectives relates to a pattern recognition system comprising a datainput unit for inputting data of a predetermined dimension, a pluralityof data processing modules having feature detection layers for detectinga plurality of features, and an output unit for outputting a result of apattern recognition, wherein the data processing module includes aplurality of arithmetic elements connected to each other by apredetermined synaptic connection unit, each of the arithmetic elementoutputs a pulsed signal at a frequency or timing corresponding to anarrival time pattern of a plurality of pulses within a predeterminedtime window, the output unit outputs, based on the outputs of theplurality of arithmetic elements, a result of detecting or recognizing apredetermined pattern, and the synaptic connection unit includes amodulation circuit for inputting the plurality of pulsed signals fromthe different arithmetic elements and effecting a predetermined commonmodulation on a plurality of predetermined pulsed signals among theplurality of pulsed signals, and outputs in branch the modulated pulsesignals to different signal lines.

According to still another aspect, the present invention which achievesthese objectives relates to a pulse signal processing circuit comprisinga modulation circuit for inputting a plurality of pulsed signals fromdifferent arithmetic elements and giving a predetermined delay to eachpulse, and a branch circuit for outputting in branch the modulated pulsesignals in a predetermined sequence to different signal linesrespectively in a way that gives a predetermined delay to the pulsesignal.

According to yet another aspect, the present invention which achievesthese objectives relates to a parallel processing circuit comprising aplurality of arithmetic elements, arrayed in parallel, for extracting adifferent feature pattern category in every predetermined area withrespect to a predetermined sampling position on input data of apredetermined dimension, wherein each of the arithmetic elements isconnected to other predetermined arithmetic element through synapticconnection unit, and the plurality of arithmetic elements for extractingthe different feature pattern category relative to the predeterminedposition on the input data, are disposed adjacent to each other.

According to yet another aspect, the present invention which achievesthese objectives relates to a pulse signal processing circuit comprisinga parallel modulation circuit for inputting a plurality of pulsedsignals from different arithmetic elements and effecting a predeterminedmodulation in parallel on a plurality of predetermined signals among theplurality of pulse signals, and an integration unit for integratingoutputs of the parallel modulation circuit, wherein the parallelmodulation circuit includes a plurality of time window integrationcircuits for effecting a predetermined weighted time window integrationwith respect to a plurality of predetermined signals among the modulatedpulse signals, and the arithmetic element outputs a predetermined pulsesignal on the basis of a signal from the integration unit.

According to a further aspect, the present invention which achievesthese objectives relates to a pulse signal processing circuit comprisinga timing signal generation circuit, a connection unit for connecting thearithmetic elements, wherein the connection unit inputs the pulsesignals from the predetermined arithmetic elements and executes apredetermined weighted time window integration, and the arithmeticelements are disposed in parallel by the connection unit and integratethe pulse modulation signals from the connection unit on the basis of atiming signal from the timing signal generation circuit.

Other objectives and advantages besides those discussed above shall beapparent to those skilled in the art from the description of a preferredembodiment of the invention which follows. In the description, referenceis made to accompanying drawings, which form a part thereof, and whichillustrates an example of the invention. Such example, however, is notexhaustive of the various embodiments of the invention, and thereforereference is made to the claims which follow the description fordetermining the scope of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a whole architecture of a network fora pattern detection/recognition system in one embodiment of the presentinvention;

FIGS. 2A, 2B and 2C are diagrams showing configurations of a synapticportion and a neuron element portion;

FIGS. 3A and 3B are diagrams showing how a plurality of pulses arepropagated to feature detection layer neurons from a feature integrationlayer or an input layer;

FIGS. 4A, 4B and 4C are diagrams showing an architecture of a synapticcircuit;

FIGS. 5A, 5B and 5C are diagrams showing an architecture of a synapticconnection small circuit, and an architecture of a pulse phase delaycircuit;

FIG. 6 is a diagram showing a network architecture when inputted to thefeature detection layer neuron from a pacemaker neuron;

FIGS. 7A, 7B, 7C, 7D and 7E are graphs showing a structure of a timewindow, an example of a weighting function distribution and an exampleof feature elements when processing a plurality of pulses correspondingto the different feature elements, which are inputted to featuredetection neurons;

FIG. 8 is a diagram showing cells on respective layers;

FIGS. 9A and 9B are diagrams each showing an example of a neuronconfiguration (array);

FIGS. 10A and 10B are diagrams showing a sharing structure of a synapticconnection circuit;

FIGS. 11A and 11B are diagrams showing another sharing structure of thesynaptic connection circuit;

FIG. 12 is a diagram showing a further sharing structure of the synapticconnection circuit;

FIG. 13 is a diagram showing a detailed architecture of the synapticconnection circuit;

FIG. 14 is a timing chart showing behavioral timings of the respectiveelements of the synaptic circuit in an embodiment 3;

FIGS. 15A and 15B are diagrams schematically showing a basicarchitecture of the network in an embodiment 4;

FIG. 16 is a diagram schematically showing how weighting coefficientsignals are distributed to the respective synaptic circuits from aweighting coefficient generation circuit; and

FIG. 17 is a diagram schematically showing an architecture of an imageinput system mounted with an object recognition system using a parallelpulse signal processing circuit in the embodiment 4.

DESCRIPTION OF THE PREFERRED EMBODIMENTS First Embodiment

One preferred embodiment of the present invention will hereinafter bedescribed in detail with reference to the accompanying drawings.

Outline of Whole Architecture

FIG. 1 is a diagram showing a whole architecture of a network for apattern detection/recognition system in the first embodiment. Thispattern detection/recognition system mainly deals with informationrelated to a recognition (detection) of an object or a geometricalfeature.

FIG. 1 illustrates a so-called convolutional network architecture(LeCun, Y. and Bengio, Y., 1995, “Convolutional Networks for ImagesSpeed, and Time Series” in Handbook of Brain Theory and Neural Networks(M. Arbib, Ed.), MIT Press, pp. 255–258). A different point from theprior arts is, however, that inter-layer connections within the sameroute can take a form of local interconnections (which will hereinafterbe described). A final output is defined as a result of the recognition,i.e., a category of the object recognized.

A data input layer 101 is a CMOS (Complementary Metal-OxideSemiconductor) sensor or a photoelectric converting device such as a CCD(Charge Coupled Device) in the case of detecting and recognizing animage, and a voice input sensor in the case of detecting and recognizinga voice. Further, the data input layer 1 may be structured to inputhigh-order data obtained from a result of analysis (for example, aprimary component analysis and so on) by a predetermined data analyzingportion.

Given hereinafter is an explanation of the case of inputting the image.A feature detection layer (1, 0) detects, based on a multiple resolutionprocess such as Gabor wavelet conversion and others, a local low-orderfeature (that may contain a color component feature in addition to thegeometrical feature) of an image patter by the number of a plurality offeature categories at a plurality of scale levels or with a plurality ofresolutions at the same area in each of positions over the entire imagesurface (or at each of predetermined sampling points over the entireimage surface). The feature detection layer (1, 0) is constructed ofneuron elements, each having a receptive field structure correspondingto a category of feature amount (which is, e.g., gradients of linesegments defined as a geometrical structure in the case of extractingthe line segments in a predetermined direction as a geometricalfeature), and generating a pulse train corresponding to a degreethereof.

A feature detection layer (1, k) configures processing channels with theplurality of resolutions (or at the plurality of scale levels) on thewhole (wherein k≧0). Namely, when exemplifying a case where the Gaborwavelet conversion is executed on the feature detection layer (1, 0), aset of feature detection cells with Gabor filter kernels having the samescale level but different directional selectivities as a receptive fieldstructure, configure the same processing channel on the featuredetection layer (1, 0), and, on a subsequent layer (1, 1) also, thefeature detection cells (for detecting a higher-order feature) receivingoutputs from the former feature detection cells, belong to the samechannel as the above processing channel. On a still subsequent layer (1,k) (wherein k>1) also, the feature detection cells receiving the outputsfrom the plurality of feature integration cells configuring the samechannel similarly on a (2, k−1) layer, are structured to belong to thischannel. The processing at the same scale level (or with the sameresolution) proceeds through each processing channel, wherein thelow-order through high-order features are detected and recognized byhierarchical parallel processing.

A feature integration layer (2, 0) has a predetermined receptive fieldstructure (a receptive field 105 hereinafter implies a connecting rangewith an output element of an immediate anterior layer, and the receptivefield structure implies a connecting weight distribution), and isconstructed of neuron elements each generating the pulse train. Thefeature integration layer (2, 0) integrates a plurality of neuronelement outputs within the same receptive field from the featuredetection layer (1, 0) (the integration involving an operation such assub-sampling based on local averaging, a maximum output extraction andso on). Further, each receptive field of the neuron within the featureintegration layer has a structure common to the neurons within the samelayer. Each of the feature detection layers 102 (1, 1), (1, 2), . . . ,(1, N) and the feature integration layers 103 ((2, 1), (2, 2), (2, N))has a predetermined receptive field structure. The former group offeature detection layers ((1, 1), . . . ) detect, as by the respectivelayers described above, a plurality of different features in respectivefeature detection modules. The latter group of feature integrationlayers ((2, 1), . . . ) integrate results of detecting the plurality offeatures from the anterior feature detection layers. The former group offeature detection layers are, however, connected (wired) to receive cellelement outputs of the anterior feature integration layers belong to thesame channel. The sub-sampling defined as a process executed by thefeature integration layer involves averaging the outputs from localareas (local receptive fields of the concerned feature integration layerneurons) from a feature detection cell group coming under the samefeature category.

FIGS. 2A, 2B and 2C are diagrams showing a configuration of a synapticcircuit and a configuration of the neuron element. A structure forconnecting inter-layer neuron elements 201 is, as depicted in FIG. 2A,built by a signal transmission portion 203 (a wire or a delay line)corresponding to an axon of a neural cell and by synaptic circuits S202corresponding to dendrites thereof. FIG. 2A shows the connectingarchitecture related to the outputs (that are inputs if viewed from acertain feature detection (integration) cell (N)) from a neuron group(n_(i)) of a feature integration (detection) cell that configures thereceptive field with respect to the feature detection (integration) cell(N). The signal transmission portion 203 drawn by a bold line serves asa common bus line, and pulse signals from the plurality of neurons,which are arranged in time-series, are transmitted through on thissignal transmission line. The same architecture is also adopted in thecase of receiving the inputs from the cell (N) as an output destination.In this case, the input signals and the output signals may be processedin separation on the time-base absolutely in the same architecture, orthe processing may be executed in a way that gives the same architectureas FIG. 2A shows in two systems for inputting (the dendrite-side) andfor outputting (the axon-side).

The synaptic circuits S202 are categorized into those related to theinter-layer connections (which are the connection between the neurons onthe feature detection layer 102 and the connection between the neuronson the feature integration layer 103, and there might exist the on-layerneuron connections to a posterior layer and to an anterior layer), andthose related to the connections between the neurons within on the samelayer. The latter type of synaptic circuits are used, as the necessitymay rise, mainly for connections with pacemaker neurons that will beexplained later on and with the feature detection or integrationneurons.

In the synaptic circuit S202, a so-called excitatory connection involvesamplifying the pulse signals, while an inhibitory connection involvesattenuating the signals. In the case of transmitting the informationthrough on the pulse signals, the amplification and the attenuation canbe actualized by any one of an amplitude modulation, a pulse widthmodulation, a phase modulation and a frequency modulation of the pulsesignal.

According to the first embodiment, the synaptic circuit S202 is usedchiefly for a pulse phase modulation element, wherein the signalamplification is converted into a substantial advance of a pulse arrivaltime as a quantity intrinsic to a feature, whereas the attenuation isconverted into a substantial delay. Namely, the synaptic connection, aswill be mentioned later on, gives an arrival position (phase) on thetime-base that is intrinsic to the feature in the neurons at the outputdestination, wherein the excitatory connection gives a phase advance ofthe arrival pulse with respect to a certain reference phase in terms ofa qualitative aspect, and the inhibitory connection gives a delaylikewise.

Referring to FIG. 2A, each of neuron elements n_(j) outputs the pulsesignals (a spiked signal train), and involves the use of a so-calledintegrate-and-fire type neuron element as will be explained below. Notethat the synaptic circuit and the neuron elements may, as illustrated inFIG. 2C, be combined to configure a circuit block.

Neuron Element

Next, the neurons that form each layer will be described. Each of theneuron elements is extension-modeled based on the so-calledintegrate-and-fire type neuron, and is the same as thisintegrate-and-fire type neuron in terms of such a point that the neuronelement fires when a result of linearly adding the input signals (apulse train corresponding to an action potential) spatiotemporallyexceeds a threshold value, and outputs the pulse signals.

FIG. 2B shows one example of a basic architecture representing abehavior principle of the pulse generation circuit (CMOS circuit)defined as the neuron element, and illustrates what a known circuit(IEEE Trans. On Neural Networks Vol. 10, p. 540) is extended. Herein,the pulse generation circuit is constructed as what receives theexcitatory input and the inhibitory input.

The behavior principle of this pulse generation circuit will hereinafterbe explained. A time constant of a capacitor C₁/resistor R₁ circuit onthe side of the excitatory input, is smaller than a time constant of acapacitor C₂/resistor R₂ circuit on the side of the inhibitory input. Ina steady state, transistors T₁, T₂, T₃ are cut off. Note that theresistor is actually constructed of a transistor connected in a diodemode.

When an electric potential of the capacitor C₁ increases and gets higherby a threshold value of the transistor T₁ than that of the capacitor C₂,the transistor T₁ becomes active, and further the transistors T₂, T₃ getactive. The transistors T₂, T₃ form a current mirror circuit, and anoutput of the circuit shown in FIG. 2B is given forth from the side ofthe capacitor C₁ by an unillustrated output circuit. The circuit isstructured so that when an electric charge accumulation amount of thecapacitor C₂ is maximized, the transistors T₁ falls into a shutdown,then the transistors T₂, T₃ are cut off as a result of the aboveshutdown, and a positive feedback comes to 0.

During a so-called refractory period, the capacitor C₂ discharges, and,if a potential of the capacitor C₁ is larger than a potential of thecapacitor C₂ and unless a difference therebetween is over the thresholdvalue of the transistor T₁, the neuron does not respond. The periodicpulses are outputted with a repetition of alternate charging/dischargingof the capacitors C₁, C₂, and a frequency thereof is generallydetermined corresponding to a level of the excitatory input. Owing to anexistence of the refractory period, the frequency can be, however,restricted to the maximum value, and a fixed frequency can also beoutputted.

The potential, i.e., the charge accumulation amount of the capacitor iscontrolled in terms of the time by a reference voltage control circuit(time window weighting function generation circuit) 204. What reflectsthis control characteristic is a weighted addition within a time windowwith respect to the input pulse, which will be mentioned later on (seeFIGS. 7A, 7B, 7C, 7D and 7E). This reference voltage control circuit 204generates a reference voltage signal (corresponding to a weightingfunction in FIG. 7B) on the basis of an input timing (or aninterconnection input to the neuron of the subsequent layer) from apacemaker neuron that will hereinafter be described.

The inhibitory input is not necessarily required in the first embodimentin some cases, however, the input to the feature detection layer neuronfrom the pacemaker neuron is set inhibitory, whereby a divergence(saturation) of the output can be prevented.

Generally, a relationship between the summation of the input signals andthe output level (the pulse phase, the pulse frequency, the pulse widthand so forth) changes depending on a sensitivity characteristic of theneuron. This sensitivity characteristic can be changed depending on atop-down input from a higher-order layer. In the following discussion,it is assumed for an explanatory convenience that circuit parameters beset so that a pulse output frequency corresponding to the summationvalue of the input signals rises steeply (therefore, the values aresubstantially binary in a frequency domain) and that the output level(such as a timing with a phase modulation added) and so on) fluctuatesdepending on the pulse phase modulation.

Moreover, a pulse phase modulation portion may have an addition of acircuit as shown in FIGS. 5A, 5B and 5C, which will hereinafter bedescribed. With this scheme, the weighting function in the time windowis controlled based on the reference voltage with the result that thephase of the pulse output from this neuron changes, and this phase canbe used as an output level of the neuron.

A time τ_(w1), as shown in FIG. 7B, corresponding to a maximum value ofthe weighting function that gives a time integrating characteristic(receiving sensitivity characteristic) with respect to the pulse havingundergone the pulse phase modulation at the synaptic connection, isgenerally set earlier in time than an arrival predicted time τ_(s1) ofthe pulse intrinsic to the feature given by the synaptic connection. Asa result, the pulse arriving earlier than the arrival predicted timewithin a fixed range (the pulse arriving too early is attenuated in theexample in FIG. 7B) is, in the neuron receiving this pulse, integratedin time as a pulse signal having a high output level. A profile of theweighting function is not limited to a symmetry as seen on Gaussianfunction etc and may assume an asymmetry. It should be noted based onthe gist elucidated above that the center of each weighting function inFIG. 7B does not correspond to the pulse arrival predicted time.

Further, an output phase of a (presynaptic) neuron has such an outputcharacteristic that a delay (phase from a fiducial time corresponding tothe beginning of the time window as will be explained later on, isdetermined by the charge accumulation amount after detecting phasesynchronization when receiving the reference pulse (based on thepacemaker output and others). A detailed architecture of the circuitgiving this output characteristic is not essential to the presentinvention and is therefore omitted herein. A pulse phase of apostsynaptic neuron is what the pulse phase of the presynaptic neuron isadded to an intrinsic phase modulation amount given at the synapseconcerned.

Further, there may also be utilized such a known circuit architecture asto give forth an oscillatory output delayed by a predetermined timingwhen the input summation value obtained by use of the window functionand so on exceeds the threshold value.

The architecture of the neuron elements using the neurons belonging tothe feature detection layer 102 or the feature integration layer 103,may take such a circuit architecture as to output the pulse with a phasedelay corresponding to the input level (the simple or weighted summationvalue of the inputs described above) at which the concerned neuronreceives from the receptive field of the anterior layer after receivingthe pulse outputted from the pacemaker neuron in a case where a firingpattern is controlled based on an output timing of the pacemaker neuronthat will be mentioned later on. In this case, before the pulse signalfrom the pacemaker neuron is inputted, there exists a transient statewhere the respective neurons output the pulses in random phases withrespect to each other in accordance with the input levels.

The neuron of the feature detection layer 102 has, as explained above,the receptive field structure corresponding to the feature category, andoutputs the pulse with an output level (given herein in the form of thephase change; it may also be structured to show a change based on thefrequency, the amplitude and the pulse width) taking a so-calledsquashing function value, i.e., such a non-reductive and nonlinearfunction as to gradually saturate with a fixed level, as in the case of,e.g., a sigmoidal function, etc., in accordance with a weight summationvalue (that will be explained below) when this weight summation valuedepending on the time window function of the input pulse signal from theneuron of the anterior layer (the input layer 101 or the featureintegration layer 103) is equal to or larger than the threshold value.

Synaptic Circuit and Others

FIGS. 4A, 4B and 4C show a matrix layout of synaptic connection smallcircuits each giving a synaptic connection strength (that implies amagnitude of the modulation in regard to the phase delay etc) to each ofneurons n′_(j) to which the neurons n_(i) are connected in the synapticcircuit 202 (S_(i))

As described above, each neuron of the feature detection layer 102 hasthe local receptive field structure (which is the local synapticconnection structure to the anterior layer) in accordance with the astructure of the pattern to be detected. This local receptive fieldstructure contains a plurality of synapses that give a symmetry or acommon connection strength. With respect to the symmetry of theconnection structure, excluding the symmetry of a connection strengthdistribution pattern as viewed from the neuron on the input side (areceiving side) of the signal, there exists a symmetry (or a pluralityof connection strengths taking the same value) of a connection strengthdistribution pattern as viewed from the neuron on the output side (atransmitting side).

The former is typically a case where the feature detection layer neuronfor detecting a certain feature category inputs, partially or totallywith the same connection strength, pulse outputs (strength levels) fromthe feature integration layer neurons (corresponding to input layerpixels) with respect to a plurality of different low-order featurecategories (or input pixel portions). For instance, each of thereceptive field structures of the feature detection layer neurons forperforming the Gabor wavelet conversion assumes the symmetry and has thesame sensitivity level (connection strength) in a plurality of positionsof the receptive fields.

The latter is a case where, for example, a plurality of pulse signalshaving undergone an intrinsic modulation in the synaptic connectionelement, are outputted to the plurality of neurons, detecting thedifferent feature categories, of the feature detection layer defined asa subsequent layer from the feature integration layer neuronrepresenting a certain category, and a synaptic connection patternassumes a distribution symmetry (or this connection pattern gives thesame modulation amount in the plurality of synaptic connections).

On the other hand, the connection to the neuron of the middle- orhigh-order feature detection layer 102 from the low- or middle-orderfeature integration layer 103 in the former structure, generally canalso take a non-local receptive field structure (connection pattern)(see FIG. 1). If laid out as shown in FIG. 8 and FIGS. 9A and 9B,however, the local structure is obtained, and the same symmetry (or theplurality of connection distribution distributions at the same levelwithin the same receptive field) with respect to the local receptivefield structure can be given.

Namely, the non-local receptive field structure described above, in thestructure depicted in FIG. 1, the feature integration layer neurons of aplurality of feature integration modules (indicated by small rectangularareas on, e.g., a (2,0) layer in FIG. 1) belonging to feature categoriesdifferent from each other, are arranged, even when related to a featureconcerning the same position on the input data, in positions that arespatially far distant from each other if the feature integration moduleto which these neurons belong differs, which means that the connectionto the feature detection layer from the feature integration layer takessuch a non-local wiring structure that the positional proximity (or thecoincidence) on the input data is not necessarily the proximity in termsof wiring because of those outputs from the plurality of featureintegration layer neurons being inputted to the feature detection layerneuron.

A structure shown in FIG. 8 and FIGS. 9A and 9B will hereinafter beexplained. The feature integration layer neurons with respect to theplurality of geometrical features in a predetermined position (or inlocal areas with this position being centered) on the input data, aredisposed adjacent to each other, and the respective neurons haveconnections to the higher-order feature detection later neurons.Referring to FIG. 8, a feature detection cell F_(D) (r, f_(k), i) isdefined as a cell for detecting an i-th feature category on a featuredetection layer k in a position corresponding to a location r on theinput data. Further, a feature integration cell F_(I) (r, f_(k), i) islikewise defined as a cell related to an i-th feature category on afeature integration layer k in the position corresponding to thelocation r on the input data. FIG. 8 schematically shows that each areashowing, together with the local receptive field, the inter-layerconnection to the feature integration layer from the feature detectionlayer with respect to the low-order feature, has the local receptivefield structure in each inter-layer connection unlike FIG. 1.

For instance, if the number of the feature categories to be extracted,is 4, the neuron elements corresponding to the respective featurecategories (F1, F2, F3, F4) are locally arrayed in cluster asillustrated on the left side in FIG. 9A. Herein, the neuron elementsarrayed in cluster represents the feature integration layer neurons withrespect to the different geometrical features in the same position onthe input data. According to the first embodiment, the featureintegration layer involves the use of an array structure as illustratedon the left side in FIG. 9A. Further, FIG. 9A schematically shows wiringfrom the unspecified neurons of the feature integration layer to ahigh-order (corresponding to a middle level in the network as a whole)feature detection layer.

On the feature detection layer, high-order feature categories (which areherein two categories G1, G2) are detected in every predeterminedposition on the input data. The neurons relative to the category G1receive outputs from the neurons (belonging to elliptical areas definedby dotted lines in FIG. 9A and related to an existence or non-existenceof this category in positions corresponding to a plurality of locationsin the local area on the input data) relative to F1, F2 of the featureintegration layer. Similarly, the neurons relative to the category G2receive outputs from the neurons related to F3, F4 of the featureintegration layer. Referring again to FIGS. 9A and 9B, the adjacentneurons of the feature detection layer receive outputs from the neurons(belonging to overlapped area segments of the ellipses in FIGS. 9A and9B) existing in overlapped area segments on the feature integrationlayer.

FIG. 9B further schematically shows a part of the structure of wiringfrom the feature detection layer to the feature integration layerdefined as a high-order layer of this feature detection layer, and alsoan array of the neuron elements of the feature integration layercorresponding thereto. Herein, feature categories (g1, g2) of therespective neurons of the feature integration layer are mapped to thefeature categories (G1, G2), respectively, and geometrical featurerepresents the same category (the representations are distinguished fromeach other for the convenience's sake). The respective featureintegration layer neurons receive the outputs from the plurality ofneurons existing in the local areas on the feature detection layer.

A circuit layout in matrix is that the signal lines of which a synapticconnection strength (phase delay amount) is common are clustered by asingle synaptic small circuit with respect to every common synapticconnection strength. To be specific, the input signal lines from theplurality of different neurons are connected to a shared synaptic smallcircuit that should exist on the input side to the unspecified featuredetection layer neurons, and further the output signal lines to theplurality of different neurons or the signal lines each to the singleneuron are connected thereto as the signal lines to an outputdestination (the feature integration layer) from the concerned neurons.

Referring to FIGS. 10 and 11, there will be explained a sharing processof the synaptic connection circuit, which is executed in a case where aconnection pattern (synaptic modulation distribution) to the respectivefeature detection layer neurons assumes a symmetry, or a case where aplurality of synaptic connections give the same modulation amount.

In common throughout the respective Figures, feature detection layerneurons M1, M2, M3 have connections in sequence through a synapticcircuit (plus switch circuit) group, wherein one connection is formed bya neuron group (N1 through N5) with N3 being centered on the featureintegration layer, another connection is formed by a neuron group (N3through N7) with N5 being centered, and a further connection is formedby a neuron group (N5 through N9) with N7 being centered. The symbolsD₁, D₂, D₃ shown in the synaptic circuits (plus switch circuits)represent pulse delay quantities at the respective synapses. Givenherein by way of a simple example is such a topology that in theconnections to the feature detection layer neurons, the output from thecentral neuron receives the delay amount D₂, the outputs from theneurons most vicinal (adjacent on both sides) to the central neuronreceive D₃, and the output from the neurons positioned at the seconddistance from the central neuron.

Referring to FIG. 10A, the respective pulse signal outputs from theplurality of neurons are given the fixed delays in the same circuits andthen outputted in branch to the different neurons in accordance with theneurons on the input side by use of the synapse plus branch circuits.The branch output implies that the pulses modulated with thepredetermined delay quantities are outputted to the plurality of featuredetection neurons, and, referring again to FIG. 10A, the branch linesare wired as shown in FIG. 9B within the synaptic circuit for giving thedelay amount D₁.

For example, the pulse output from the feature integration layer neuronN2 is given the delay amount D₁ in the synaptic circuit and thereafteroutputted in branch to only the feature detection layer neuron M1.Herein, as shown in FIG. 9B, diodes are set on the branch wiresposterior to the delay element, whereby the outputs from the specifiedneurons are outputted to the specified neurons owing to the branchstructure. The output from the neuron N4 is, after being given the delayD₁, outputted to the feature detection layer neurons M1, M2. This isbecause the receptive fields of the neurons M1, M2 are overlapped at N4.Further, the pulse outputs from N5 are a signal outputted to M2 afterreceiving the delay amount D₂ and a signal outputted to M1 afterreceiving the delay D₃. The delays of the outputs from the neurons andthe detailed branch output structure are the same as those shown in FIG.11A.

FIG. 11A shows that a small circuit D_(ij) in each synaptic circuitgives a delay amount D_(i) to the pulse and outputs it to the neuron Mj.Further, the inputs to the respective synaptic circuits giving the delayamount D_(i) require a larger amount of wires for connecting the smallcircuits different depending on the branch output destinations than inFIGS. 10A and 10B.

As the input signal lines to the unspecified feature integration layerneurons, in the case of implementing the local averaging of uniformweighting for sub-sampling with respect to these neurons, as will beexplained later on, the synaptic circuit for giving the predeterminedpulse delays etc are not required to be provided midway. In the case ofexecuting a process such as non-uniform local averaging etc, however,there may be taken the same architecture of the synaptic connectioncircuit to the feature detection layer neuron.

Each of those output signal lines may be connected to a predeterminedsynaptic circuit or may also be connected as a simple branch line (delayline or wire) to an output destination neuron. Note that the signal, itis assumed, be a pulse signal inputted and outputted in a voltage mode.

If the network takes such an architecture as to have a shared connectionmode (for representing the synaptic connection structures of theplurality of neurons with the weighting coefficient distribution givenin 1) of the connection weights, the delay amount (P_(ij) given in thefollowing formula (1)) at each synapse can be uniformed within the samereceptive field in some cases unlike the case in FIGS. 3A and 3B. Forexample, the connection to the feature integration layer from thefeature detection layer, if the feature integration layer performssub-sampling based on the local averaging (which is to be, however, theuniform weighting) of the outputs of the feature detection layer definedas an anterior layer thereto, may take the above architecture withoutdepending on the detection object (i.e., without depending on a categoryof the object).

In this case, as illustrated in FIG. 4C, a single circuit S_(k,i)suffices for forming each of the synaptic connection small circuits 401in FIG. 4A, and this circuit architecture is particularly economical. Onthe other hand, if the connection to the feature detection layer fromthe feature integration layer (or a sensor input layer) takes thiscircuit architecture, what the feature detection neuron detects is suchan event that the pulses representing a plurality of different featureelements arrive simultaneously (or arrive substantially at the sametime).

As depicted in FIG. 4B, each of the synaptic connection small circuits401 is constructed of a learning circuit 402 and a phase delay circuit403. The learning circuit 402 adjusts the above delay amount by changinga characteristic of the phase delay circuit 403. Further, the learningcircuit 402 stores a characteristic value thereof (or a control valuethereof) on a floating gate element or on a capacitor connected to thefloating gate element. The phase delay circuit 403 is classified as apulse phase modulation circuit and is, as shown in FIG. 5A, configuredby using, for instance, monostable multivibrators 506, 507, resistors501, 504, capacitors 503, 505 and a transistor 502. FIG. 5B showsrespective timings of a rectangular wave P1 ([1] in FIG. 5B) inputted tothe monostable multivibrator 506, a rectangular wave P2 ([2] in FIG. 5B)outputted from the monostable multivibrator 506, and a rectangular waveP3 ([3] in FIG. 5B) outputted from the monostable multivibrator 507.

Though a detailed explanation of an operational mechanism of the phasedelay circuit 403 is omitted, a pulse width of the rectangular wave P1is determined by a time till a voltage of the capacitor 503 based on acharging current reaches a predetermined threshold value, while a pulsewidth of the rectangular wave P2 is determined by a time constant of theresistor 504 and the capacitor 505. If the pulse width of P2 expands (asindicated by a dotted-line rectangular wave in FIG. 5B) and if a falltiming thereof is shifted back, a rise time of P3 is shifted by the samequantity, however, the pulse width of P3 remains unchanged, and ittherefore follows that the rectangular wave is outputted in a way ofbeing modulated by a phase of the input pulse.

A control voltage Ec is changed by the learning circuit 402 forcontrolling the charge accumulation amount to a refresh circuit 509having the reference voltage and to the capacitor 508 for giving theconnection weight, whereby the pulse phase (delay amount) can becontrolled. A long-term retainment of this connection weight may involvestoring the connection weight as charge of the floating gate element(not shown) provided outside the circuit shown in FIG. 5A after thelearning behavior or by writing it to a digital memory and so on. Theremay be utilized other known circuit architectures such as thearchitectures (refer to e.g., Japanese Patent Application Laid-Open Nos.5-37317 and 10-327054) each schemed to downsize the circuit.

What is exemplified as the learning circuit at the synapse thatactualizes the simultaneous arrival of the pulses or the predeterminedphase modulation amount, includes the circuit elements as shown in FIG.5C. To be specific, the learning circuit 402 can be constructed of apulse propagation time measuring circuit 510 (a propagation time hereinindicates a time difference between a time of the pulse output of apresynaptic neuron on a certain layer and an arrival time of this pulseat an output destination neuron existing on a next layer), a time windowgeneration circuit 511, and a pulse phase modulation amount adjustingcircuit 512 for adjusting a pulse phase modulation amount in thesynaptic portion so that the propagation time takes a fixed value.

The propagation time measuring circuit 510 involves the use of anarchitecture for inputting clock pulses from the pacemaker neuronsconfiguring the same local receptive field as will be explained later onand obtaining the propagation time based on an output from a countercircuit for these clock pulses in duration of a predetermined time width(time window: see FIG. 3B). Note that the time window is set based on apoint of firing time of the output destination neuron, whereby Hebb'slearning algorithm (rule) extended as shown below is applied. Process(Extraction of Low-Order Feature by Gabor Wavelet conversion etc) onFeature Detection Layer (1, 0)

Supposing that the feature detection layer (1,0) contains the neuronsdetecting a structure (low-order feature) of a pattern having apredetermined spatial frequency in a local area having a certain sizeand a directional component of being vertical and if there exists astructure corresponding to an interior of the receptive field of N1 onthe data input layer 1, the neuron outputs the pulse in phasecorresponding to a contrast thereof. This type of function can beactualized by a Gabor filter. A feature detection filter functionperformed by each of the neurons of the feature detection layer (1,0)will hereinafter be discussed.

It is assumed that the Gabor wavelet conversion expressed by a filterset having multi-scales and multi-directional components on the featuredetection layer (1,0) and each of the intra-layer neurons (or each groupconsisting of a plurality of neurons) has a predetermined Gaborfiltering function. On the feature detection layer, one single channelis configured by clustering a plurality of neurons groups eachconsisting of neurons having the receptive field structurescorresponding to a convolutional operation kernels of a plurality ofGabor functions that have a fixed scale level (resolution) and differentdirectional selectivities. On this occasion, the neuron group formingthe same channel has a different directional selectivity, and the neurongroups exhibiting the same size selectivity may be disposed in positionsadjacent to each other, or the neuron groups belonging to differentprocessing channels may also be disposed adjacent to each other. Thisscheme is based on an idea that the actualization is easier in terms ofthe circuit architecture by adopting the layouts shown in the respectiveFigures for the convenience's sake of a connecting process that will bementioned below in the group-oriented coding.

Incidentally, for details of the method of executing the Gabor waveletconversion in the neural network, refer to a document (IEEE Trans. OnAcoustics, Speed, and Signal Processing, vol. 36, pp. 1169–1179) byDaugman (1988).

Each of the neurons of the feature detection layer (1,0) has thereceptive field structure corresponding to a kernel g_(mn). The kernelg_(mn) having the same scale index m has a receptive field of the samesize, and a corresponding kernel g_(mn) size is set corresponding to thescale index in terms of the operation. Herein, the sizes such as 30×30,15×15 and 7×7 are set on the input image in sequence from the roughestscale. Each neuron outputs the pulse at such an output level (which isherein on a phase basis; an architecture on a frequency basis or anamplitude basis or a pulse basis may also, however, be used) as tobecome a nonlinear squashing function of a wavelet conversioncoefficient value obtained by inputting a sum of products ofdistribution weighting coefficients and image data. As a result, itfollows that the Gabor wavelet conversion is executed as an output ofthis whole layer (1,0).

Processes (Extractions of Middle- and High-Order Features) on FeatureDetection Layer

Unlike the feature detection layer (1,0), each of the neurons of thesubsequent feature detection layers ((1,1), (1,2), . . . ) forms, basedon the so-called Hebb's learning algorithm etc, the receptive fieldstructure for detecting a feature intrinsic to a pattern of arecognition object. On a more posterior layer, a size of the local areain which to detect the feature becomes stepwise more approximate to asize of the whole recognition object, and geometrically a middle- orhigh-order feature is detected.

For instance, when detecting and recognizing a face, the middle- (orhigh-order) feature represents a feature at pattern-element-orientedlevels such as eyes, a nose, a mouth etc shaping the face. Betweendifferent channels, if at the same hierarchical level (the same level interms of a complexity of the feature to be detected), a difference ofthe feature detected comes under the same category but is what isdetected by the scales different from each other. For example, the (eye)defined as the middle-order feature is detected as an (eye) having adifferent size at a different processing channel. Namely, the scheme isthat the in-image (eye) having a given size is detected at the pluralityof processing channels exhibiting different scale level selectivities.

Note that each of the neurons of the feature detection layer maygenerally have such a mechanism as to receive, based on the output ofthe anterior layer, an inhibitory (shunting inhibition) connection inorder to stabilize the output (without depending on the extractions ofthe low- and high-order features).

Process on Feature Integration Layer

The neurons of the feature integration layers ((2,0), (2,1), . . . )will be explained. As illustrated in FIG. 1, the connection to thefeature integration layer (e.g., (2,0)) from the feature detection layer(e.g., (1,0)) is configured to receive inputs of the excitatoryconnections from the neurons of the same category (type) of featureelements of the anterior feature detection layer within the receptivefields of the concerned feature integration neurons. The function of theneuron of the integration layer is, as explained above, the localaveraging for every feature category, the sub-sampling based on themaximum value detection, and so on.

According to the former mode, the plurality of pulses of the samecategory of feature are inputted, and then integrated and averaged inthe local area (receptive field) (alternatively, a representative valuesuch as a maximum value is calculated within the receptive field),thereby making it possible to surely detect a positional fluctuation anda deformation of the feature. Therefore, the receptive field structureof the neuron of the feature integration layer may be formed so as tobecome uniform (such as being in a rectangular area having apredetermined size in any cases and exhibiting a uniform distribution ofthe sensitivity or the weighting coefficient therein) without dependingon the feature category.

Pulse Signal Processing on Feature Integration Layer

As discussed above, according to the first embodiment, the featureintegration cell is not structured to receive the timing control fromthe pacemaker neuron on the feature detection layer with a layer number(1,k) anterior thereto. The reason is that in the feature integrationcell, the neurons output the pulses in phase (any one of the frequency,the pulse width and the amplitude may be dependent, however, the phaseis adopted in the first embodiment) determined not by the arrival timepattern of the input pulse but by, if anything, an input level (such asa temporal summation value of the input pulses) within a fixed timerange, and hence a time window occurrence timing is not so important.Note that this does not intend to exclude an architecture in which thefeature integration cell receives the timing control from the pacemakerneuron on the anterior feature detection layer, and this architectureis, as a matter of course, feasible.

Behavior Principle of Pattern Detection

Next, pulse encoding of a two-dimensional graphic pattern and adetection method thereof will be explained. FIGS. 3A and 3Bschematically show how the pulse signals are propagated to the featuredetection layer from the feature integration layer (e.g., from the layer(2,0) to the layer (1,1) in FIG. 1). The neurons n_(i) on the side ofthe feature integration layer correspond to feature amounts (or featureelements) different from each other, while the neurons n′_(j) on theside of the feature detection layer get involved in detecting ahigher-order feature (pattern elements) obtained by combining therespective features within the same receptive field.

An intrinsic delay (intrinsic to the feature) due to a pulse propagationtime and a time delay etc in the synaptic connection (S_(j,i)) to theneuron n′_(j) from the neuron n_(i), occurs in each inter-neuronconnection. As a result, so far as the pulses are outputted from theneurons of the feature integration layer, pulses of a pulse train Pi areset to arrive at the neuron n′_(j) in a predetermined sequence (such asP₄, P₃, P₂, P₁ in FIG. 3A), depending on a delay amount at the synapticconnection that is determined by learning.

FIG. 3B shows a pulse propagation timing to a certain feature detectioncells (n′_(j)) (detecting a higher-order feature) in a layer having alayer number (1, k+1) from feature integration cells n₁, n₂, n₃(individually representing different categories of features) in a layerhaving a layer number (2, k) in the case of executing thesynchronization control of the time window by using the timing signalsfrom the pacemaker neurons that will be mentioned later on.

Referring to FIG. 6, the pacemaker neurons n_(p) which accompany thefeature detection neurons (n_(j), n_(k) etc) for detecting differentcategories of features, form the same receptive field as that of thefeature detection neurons and receives the excitatory connection fromthe feature integration layer (or the input layer). Then, the pulses areoutputted to the feature detection neurons and the feature integrationneurons at a predetermined timing (or frequency) determined by an inputsummation value (or an activity level average value of the wholereceptive field in order to control so as to depend on a state ofrepresenting an action characteristic intrinsic to the whole receptivefield).

Further, the scheme in each feature detection neuron is that the timewindows are phase-locked to each other with its input serving as atrigger signal but are not phase-locked before receiving the input fromthe pacemaker neuron as described above, and each neuron outputs thepulse with a random phase. Further, in the feature detection neuron, atime window integration that will be explained below is not performedbefore receiving the input from the pacemaker neuron but is performed,which is triggered by the pulse input from the pacemaker neuron.

Herein, the time window, which is determined for every feature detectionlayer neuron (n′_(i)), is common to the respective neurons within thefeature integration layer forming the same receptive field with respectto the concerned cell and to the pacemaker neuron, and gives a timerange for a time window integration.

The pacemaker neuron existing on the layer having a layer number (1, k)(where k is a natural number) outputs the pulse output to each featureintegration cell of the layer having a layer number (2, k−1) and thefeature detection cell (the layer number (1, k) to which the pacemakerneuron belongs, whereby the feature detection cell gives a timing signalfor generating the time window when adding the inputs in time aspect. Astart time of this time window serves as a reference time for measuringan arrival time of the pulse outputted from each feature integrationcell. Namely, the pacemaker neuron gives the timing for outputting thepulse from the feature integration cell, and a reference pulse for atime window integration in the feature detection cell.

Each pulse is given a predetermined quantity of phase delay when passingvia the synaptic circuit, and arrives at the feature detection cellfurther via the signal transmission line such as the common bus. Asequence of the pulse train on the time-base at this time is expressedsuch as pulses (P₁, P₂, P₃) drawn by the dotted lines on the time-baseof the feature detection cell.

In the feature detection cell, if larger than the threshold value as aresult of the time window integration (normally the integration iseffected once; there may also be, however, executed the electric chargeaccumulation involving the time window integration effected multipletimes or the averaging process involving the time window integrationeffected multiple times) of the respective pulses (P₁, P₂, P₃), a pulseoutput (P_(d)) is outputted based on a termination time of the timewindow. Note that the on-learning time window shown in the same Figureis what is referred to when executing the learning algorithm that willhereinafter be discussed.

Spatiotemporal Integration of Pulse Outputs and Network Characteristic

Next, an arithmetic process of spatiotemporal weighting summation (aweight summation) of the input pulses will be explained. As shown inFIG. 7B, each neuron takes a weight summation of the input pulses by useof a predetermined weighting function (e.g., Gaussian function) forevery sub time window (timeslot), and the summation of weights iscompared with a threshold value. The symbol τ_(j) represents a centralor peak position of the weighting function of a sub time window j, andis expressed by a start time reference (an elapse time since the starttime) of the time window. The weighting function is generally a functionof a distance (a deviation on the time-base) from a predeterminedcentral position (representing a pulse arrival time in the case ofdetecting a detection target feature), and assumes a symmetry.Accordingly, supposing that the central position τ of the weightingfunction of each sub time window (timeslot) of the neuron corresponds toa time delay after learning between the neurons, a neural network forobtaining the spatiotemporal weighting summation (the weight summation)of the input pulses can be defined as one category of a radial basisfunction network (which will hereinafter be abbreviated to RBF) in thetime-base domain. A time window F_(Ti) of the neuron ni using Gaussianfunction as a weighting function is given by:

$\begin{matrix}{F_{Ti} = {\sum\limits_{j}^{N}\;{b_{ij}{\delta\left( {t - \tau_{ij}} \right)}{\exp\left( {- \frac{\left( {t - \tau_{ij}} \right)^{2}}{\sigma_{ij}^{2}}} \right)}}}} & (1)\end{matrix}$

where σ is a spread with respect to every sub time window, and b_(ij) isa coefficient factor.

Note that the weighting function may take a negative value. For example,if a certain feature detection layer neuron is to detect eventually atriangle and when detecting a feature (F_(faulse)) that is notapparently an element configuring this graphic pattern, a connectionfrom the feature detection (integration) cell and a weighting functionmaking a negative contribution can be given from pulses corresponding tothe concerned feature (F_(faulse)) in the summation value calculationprocess of the input so that the detection of the triangle is noteventually outputted even if there is a large contribution from otherfeature elements.

A spatiotemporal summation X_(i)(t) of the input signals to the neuronsn_(i) of the feature detection layer is given by:

$\begin{matrix}{{X_{i}(t)} = {\sum\limits_{j}\;{S_{ij}{F_{Ti}(t)}{Y_{j}\left( {t - \tau_{ij} - ɛ_{j}} \right)}}}} & (2)\end{matrix}$where ε_(j) is an initial phase of the output pulse from the neuronn_(j). If converged at 0 due to synchronization firing with the neuronn_(i), or if the phase of the time window is forcibly synchronized with0 due to the timing pulse input from the pacemaker neuron, ε_(j) may beset to 0 at all times. When obtaining the weight summation on the basisof the pulse input in FIG. 7A and the weighting function shown in FIG.7B, a time-varying transition of the weight summation value as shown inFIG. 7E is obtained. The feature detection layer neuron outputs thepulse when this weight summation value reaches a threshold value (Vt).

The output pulse signal from the neuron n_(i) is, as explained above,outputted to the neuron of the high-order layer with a time delay(phase) given by learning at such an output level as to become asquashing nonlinear function of the spatiotemporal summation (aso-called input summation) of the input signals (wherein the pulseoutput takes a fixed frequency (binary) and is outputted in a way thatadds a phase modulation quantity serving as the squashing nonlinearfunction with respect to the spatiotemporal summation of the inputsignals to a phase corresponding to a fixed delay amount determined bylearning).

Learning Algorithm

The learning circuit 402 may be structured so that the time windowdescribed above comes to have a narrower width as the frequency at whichan object having the same category is presented becomes larger. Withthis contrivance, the learning circuit 402 behaves so as to get close toa coincidence detection mode for detecting simultaneous arrivals of theplurality of pulses as the pattern category is more familiar (whichmeans a larger presentation count and a larger learning count). Thisscheme makes it possible to reduce the time required for detecting thefeature (to perform an instantaneous detection behavior) but is unsuitedto a fine comparative analysis of the spatial layout of the featureelements and to a distinction between the similar patterns and so forth.

In the learning process of the delay amount, for example, by extendingto a complex number domain, a complex connection weight C_(ij) betweenthe feature detection layer neuron n_(i) and the feature integrationlayer neuron n_(j) is given such as:C _(ij) =S _(ij) exp(iP _(ij))  (3)where the first i in the function exp represents an imaginary numberunit, S_(ij) denotes a connection strength, P_(ij) indicates a phase.The phase P_(ij) is a phase corresponding to the time delay of the pulsesignal outputted to the neuron i from the neuron j at a predeterminedfrequency. The connection strength S_(ij) reflects the receptive fieldstructure of the neuron i, and has a structure that generally differscorresponding to a recognition/detection object. This is separatelyformed by learning (supervised learning or self-organization), or isformed as a predetermined structure.

On the other hand, the learning algorithm for the self-organizationrelative to the delay amount is given by:

$\begin{matrix}{C_{ij}^{\&} = {{\beta\; S_{ij}\exp\left\{ {{- {i2}}\;\pi\;\tau_{ij}} \right\}} - C_{ij}}} & (4)\end{matrix}$where C is a time differential of C, τ_(ij) is the time delay (a presetquantity) described above, and β (through 1) indicates a constant. Whensolving the above equation, C_(ij) converges at β exp(−2πiτ_(ij)), andhence P_(ij) converges at −τ_(ij). Explaining an example of applying thelearning algorithm with reference to the on-learning time window shownin FIG. 3B, only when both of presynaptic neurons (n1, n2, n3) andpostsynaptic neurons (feature detection cells) fire in a time range ofthe learning time window, the connection weight is updated based on theformula (4). Note that the feature detection cells fire after an elapseof the time window in FIG. 3B and may also fire before the elapse of thetime window in FIG. 3B.

Process on Feature Detection Layer

Processes (for learning and recognition) executed mainly on the featuredetection layer will hereinafter be described. Each feature detectionlayer inputs the pulse signals with respect to a plurality of differentfeatures from the same receptive field within the processing channel setat every scale level as explained above, and calculates thespatiotemporal weighting summation (the weight summation) and implementsa threshold process. The pulse corresponding to each feature amountarrives at a predetermined time interval, depending on a delay amount(phase) predetermined by learning.

Learning control of this pulse arrival time patter is not essential tothe first embodiment and is not therefore explained in depth. Forinstance, however, to be brief, the pulse corresponding to the featureelement among the plurality of future elements configuring a certaingraphic patter, if most contributory to detecting this pattern, arrivesearlier, and, between the feature elements showing, if intact,substantially the same pulse arrival time, there is introduced acompetitive learning scheme that the pulses arrive away by a fixedquantity in time from each other. Alternatively, there may be taken sucha scheme that the pulses arrive at time intervals different betweenpredetermined feature elements (configuring a recognition object andconceived important in particular such as a feature exhibiting a largemean curvature, a feature exhibiting a high rectilinearity and soforth).

According to the first embodiment, each of the neurons corresponding tothe respective low-order feature elements within the same receptivefield on a certain feature integration layer defined as a anteriorlayer, synchronously fires (pulse output) in a predetermined phase.Generally, there exist the connections to the feature detection neurons,defined as the neurons of the feature integration layer, for detecting,though different in their positions, the same high-order feature (inthis case, there are the connections, configuring, though difference intheir receptive fields, the same high-order feature). At this time, as amatter of course, the synchronous firing occurs also among these featuredetection neurons. The output level thereof (a phase basis is hereintaken; an architecture taking a frequency basis or an amplitude basis ora pulse width basis may also, however, be adopted) is, however,determined by a summation (or average etc) of contributions from theplurality of pacemaker neurons that are each given for every receptivefield of the feature detection neuron. In the interior of the timewindow, the weighting function has such a profile that the peak valuecorresponds to a synaptic weight value. Further, a means for actualizingthe weighting addition within the time window taking the peak value isnot limited to the neuron element circuit shown in FIGS. 2A, 2B and 2Cand may be, as a matter of course, actualized otherwise.

This time window corresponds more or less to a time zone excluding therefractory period of the neuron. Namely, there is no output from theneuron even by receiving whatever input during the refractory period (atime range other than the time window), however, the behavior that theneuron fires corresponding to the input level in the time windowexcluding the time range, is similar to that of the actual biologicalneuron. The refractory period shown in FIG. 3B is a time zone fromimmediate after the firing of the feature detection cell to a start timeof the next time window. A length of the refractory period and a widthof the time window can be, of course, arbitrarily set, and therefractory period may not be set shorter than the time window as shownin FIG. 3B.

According to the first embodiment, as schematically shown in FIG. 6, thealready-explained mechanism is that the start timing described above ismade common by means of inputting the timing information (clock pulse)by the pacemaker neuron (pulse output at a fixed frequency) receivingthe inputs from the same receptive field with respect to, for example,every feature detection layer neuron.

If configured in this fashion, the synchronization control (even ifnecessary) of the time window does not need effecting throughout thenetwork, and, even when the clock pulse fluctuates as described above,the reliability of detecting the feature is not degraded because ofreceiving uniformly an influence of the output from the same localreceptive field (the on-the-time-base positional fluctuation of thewindow function becomes the same among the neurons forming the samereceptive field). A tolerance of scatter in circuit element parameteralso increases in order for the local circuit control to enable thesynchronization behavior with a reliability to be attained.

For simplicity, the feature detection neuron for detecting the triangleas a feature will be described. It is assumed that the featureintegration layer anterior thereto reacts to a graphical feature(feature elements) such as L-shaped patterns (f₁₁, f₁₂, . . . ) havingmultiple directions, combinational patterns (f₂₁, f₂₂, . . . ) of linesegments each having a continuity (connectivity) to the L-shaped patternand combinations (f₃₁, . . . ) of a part of two sides configuring thetriangle as depicted in FIG. 7C.

Further, f₄₁, f₄₂, f₄₃ shown in FIG. 7C represent features shaping thetriangles having different directions and corresponding to f₁₁, f₁₂,f₁₃. The intrinsic delay amount is set between the neurons forming theinter-layer connection by learning, and, as a result of this, in thetriangle feature detection neuron, the pulses corresponding theprincipal and different features shaping the triangle are set beforehandto arrive at respective sub time windows (timeslots) (w₁, w₂, . . . )into which the time window is divided.

For instance, the pulses corresponding to combinations of the featuresets each shaping the triangle on the whole as shown in FIG. 7A, arrivefirst at the sub time windows w₁, w₂, . . . , w_(n) into which the timewindow is divided by “n”. Herein, the delay quantities are set bylearning so that the L-shaped patterns (f₁₁, f₁₂, f₁₃) arrive at withinw₁, w₂, w₃, respectively, and the pulses corresponding to the featureelements (f₁₁, f₁₂, f₁₃) arrive at within w₁, w₂, w₃, respectively.

The pulses corresponding to the feature elements (f₃₁, f₃₂, f₃₃) arrivein the same sequence. In the case shown in FIG. 7A, the pulsecorresponding to one feature element arrive at the single sub timewindow (timeslot). The division into the sub time windows has such asignificance that an integration mode when integrating those features,e.g., a processing mode such as setting a condition that all the featureelements be detected or a condition that a given proportion of featuresbe detected and so on, is to be enhanced in its changeability andadaptability by individually surely detecting the pulses (detection ofthe feature elements) corresponding to the different feature elementsdeveloped and expressed on the time-base in the restive sub timewindows.

For instance, under conditions where the recognition (detection) objectis a face and a search (detection) for an eye defined as one of partsconfiguring the face is important (a case where the priority ofdetecting the eye's pattern is set high in the visual search), areaction selectivity ((a detection sensitivity to a specified feature)corresponding to a feature element patter selectively configuring theeye can be enhanced by introducing a feedback connection from ahigh-order feature detection layer. This scheme makes it possible todetect the feature in a way that gives a higher importance to alower-order feature element shaping a high-order feature element(pattern).

Further, assuming that the pulse corresponding to a more importancefeature is set previously to arrive at the earlier sub time window, thefeature exhibiting the higher importance is easier to detect by settinga weighting function value in the concerned sub time window larger thanvalues in other sub time windows. This importance (the detectionpriority among the features) is acquired by learning or may also bepredefined.

Accordingly, if on condition that there occurs an event such asdetecting a given proportion of feature elements, the division into thesub time windows comes to have almost no meaning, and the processing maybe implemented in one single time window.

Note that the pulses corresponding to the plurality (three) of differentfeature elements arrive respectively and may also be added (FIG. 7D).Namely, it may be based on a premise that the pulses corresponding tothe plurality-of feature elements (FIG. 7D) or an arbitrary number offeature elements, be inputted to one single sub time window (timeslot).In this case, referring to FIG. 7D, the pulses corresponding to otherfeature elements f₂₁, f₂₃ supporting the detection of an apex angleportion f₁₁, of the triangle, arrive at the first sub time window.Similarly, the pulses corresponding to other feature elements f₂₂, f₃₁supporting the detection of an apex angle portion f₁₂ arrive at thesecond sub time window.

Note that the number of divisions into the sub time windows (timeslots),the width of each sub time window (timeslot), the feature class, and theallocation of the time intervals of the pulses corresponding to thefeature elements, are not limited to those described above and can be,as a matter of course, changed.

Second Embodiment

According to a second embodiment, as shown in FIG. 11B, a branch circuitfor outputting in branch the outputs from the synaptic circuits is setas a characteristic component by use of a local timing generationelement (or pacemaker neuron) PN as shown in FIG. 11B. The branchcircuit has a demultiplexer-wise function as will be exemplified in anembodiment that follow. The branch circuit outputs the output from eachof the synaptic circuits to each of the different feature detectionlayer neurons in accordance with the timing signal transmitted from thelocal timing generation element. This scheme enables farther wires fromthe synaptic circuits to be more simplified than in the architecture inthe embodiment 1.

For instance, supposing that the timing generation element generates alow-frequency pulse signal having a smaller pulse width than a period ofthe pulse train, the branch circuit is structured to output, to thefeature integration layer neuron intrinsic to the timeslot, the pulsesignal from the synaptic circuit that is inputted to within each of thetimeslots obtained by equally dividing a time width up to a next pulserise from a fiducial time corresponding to a rise time of each pulsesignal by the number of output destination neurons. For attaining this,the branch circuit may suffice on condition that it has a switch elementfor establishing a connection to a line different for, e.g., everytimeslot and functions as a demultiplexer (refer to a third embodiment).Note that the process of the neuron receiving the signal havingundergone the modulation in the synaptic circuit, is the same as in theembodiment 1.

Third Embodiment

A third embodiment involves the use of a circuit, as a synaptic circuit,incorporating a demultiplexer function together that sets variable amodulation amount at the delay element of the synaptic circuit, controlsthe modulation (delay) amount of the pulse signal on the basis of thetiming signal from the local timing element and branches by switchingthe output pulses. With this architecture adopted, by contrast with thearchitecture in the embodiment discussed above in which the synapticcircuit giving the different delay amount is formed independently as thecircuit that is at least physically different, even such a synapticcircuit is shared in a time-sharing manner, thereby scheming to furtherdownsize the circuit scale. Note that the process of the neuronreceiving the signal having undergone the modulation in the synapticcircuit is the same as in the embodiment 1.

As shown in FIG. 12, a synaptic circuit S as a simplified version of thecircuit architecture in FIG. 11B provides branch output destinations ofthe signals after being modulated and modulation amounts (delay amounts)each different for every timeslot (see FIG. 8) obtaining by effectingtime-sharing of a period up to a next timing pulse on the basis of thefiducial time corresponding to a rise time of the timing pulse from thelocal timing element.

The synaptic circuit is structured to include, as shown in FIG. 13, atimeslot pulse generation circuit 1301, a delay amount modulationcircuit 1302, and a demultiplexer circuit 1303 constructed of a switcharray, an activation circuit etc.

The demultiplexer circuit 1303 has a characteristic function ofoutputting, with a master clock pulse timing serving as a fiducial time,inputted data pulse signals to different branch lines in a predeterminedsequence when a switch intrinsic to each of timeslots (T₁, T₂, . . . ,T_(n)) is switched ON.

FIG. 14 is a timing chart schematically showing an example of behaviorsof the respective elements of the synaptic circuit S when thedemultiplexer circuit 1303 includes four sets of switch arrays and fourlengths of output lines. Referring to FIG. 14, respective delay amounts(D₁, D₂, D₃) correspond to the modulation amounts in the respectivesynaptic circuits in the preceding embodiment, and the pulse signalafter being modulated is outputted to the branch line as an outputdestination when each switch comes to the ON-state.

The timeslot pulse generation circuit 1301, after inputting thereference timing pulse from the local timing element described above,generates the pulse signals at an interval of a predetermined time thatis shorter than a master cock interval in a way that synchronizes thereference timing pulse as a master clock (refer to the output of thetimeslot pulse generation element in FIG. 14), wherein a time widthbetween the respective pulses corresponds to the timeslot.

The delay amount modulation circuit 1302 includes a selector forinputting the master clock from the local timing element described aboveand selecting, based on the master clock, one of a plurality of delaytimes preset in the input timing sequence of the pulse signals from thetimeslot pulse generation element. Namely, each of the time-varyingdelay amounts has a certain magnitude quantized as shown in FIG. 8, andthe delay amount modulation circuit 1302 time-controls the delay amountby selecting one of the plurality of preset fixed delay circuits forevery timeslot. Then, the demultiplexer circuit 1303 branch-outputs thepulse signals each having undergone the delay modulation for everytimeslot to the neurons different from each other. Note that the delayamount may be, as a matter of course, time-controlled by methods otherthan the above-mentioned.

According to the embodiments discussed above, the pulse signalprocessing circuit includes the modulation circuit for inputting theplurality of pulsed signals from the different arithmetic elements andmodulating in common the predetermined signals among the plurality ofpulse signals, and the modulated pulse signals are outputted in branchto the different signal lines. This architecture yields such an effectthat the modulation circuit, which should provide the predeterminedmodulation amount to the plurality of pulsed signals, is not required tobe set for every pulse (every input-side signal line) in the signalprocessing system for transmitting the pulsed signals between theplurality of arithmetic elements in a way that executes thepredetermined modulation on the pulsed signals, and the circuit scalecan be downsized.

Further, in the parallel processing circuit including the plurality ofneuron elements and the synaptic connection elements for connecting theneuron elements, the synaptic connection element is constructed by useof the pulse signal processing circuit, whereby the parallel signalprocessing circuit using the pulse signals can be simplified.

Moreover, in the pattern recognition system including the data inputunit of a predetermined dimension, the plurality of data processingmodules and the data output unit for outputting the result of thepattern recognition, the data processing module has the featuredetection layer for detecting the plurality of features and isconstructed of the plurality of arithmetic elements connected to eachother by the predetermined synaptic connection unit, the arithmeticelements within the data processing module output the pulsed signals atthe frequency or timing corresponding to the arrival time pattern of theplurality of pulses within the predetermined time window, the outputunit outputs the result of detection or recognition of the predeterminedpattern on the basis of the outputs from the respective arithmeticelements on the processing layer, the synaptic connection unit includesthe modulation circuit for inputting the plurality of pulsed signalsfrom the different arithmetic elements and modulating in common thepredetermined signals among the respective pulses, the synapticconnection unit outputting in branch the respective modulated pulsedsignals to the different signal lines, and the circuit scale can bethereby downsized and simplified in the hierarchical signal processingsystem for extracting the feature on the basis of the spatiotemporaldistribution of the pulsed signals.

Further, the parallel processing circuit has the structure in which theplurality of synaptic connections are shared as one single circuit onthe basis of the distribution symmetry of the synaptic connection to thepredetermined neuron elements, thereby attaining the reduction in thecircuit scale in accordance with the degree of symmetry of the synapticconnection distribution.

Further, the modulation circuit is structured as the delay circuit forgiving the predetermined delay to the pulse signal, whereby theindividual synaptic connection weight can be actualized by the commonpulse signal delay circuit.

Still further, the pulse signal processing circuit includes themodulation circuit for inputting the plurality of pulsed signals fromthe different arithmetic elements and effecting the predeterminedmodulation on each pulse, and the branch circuit for outputting inbranch the modulated pulsed signals to the different signal lines in thepredetermined sequence in a way that gives the predetermined delay tothe pulsed signal. With this architecture, the pulsed signals havingundergone the common modulation can arrive at the plurality ofarithmetic elements (neurons) existing at the spatially different pulsepropagation distance from a certain arithmetic element (neuron) at thesame timing or the timing based on the predetermined rule.

Fourth Embodiment

According to a fourth embodiment, in the convolutional networkarchitecture already explained above, a synaptic circuit blockimplements a parallel pulse signal process including the time windowintegration with respect to the output pulsed signals from therespective feature integration layer neurons. According to theembodiments discussed so far, the time window integration is performedon the side of the neuron. By contrast, however, the time windowintegration is executed in parallel on the side of the synapse accordingto the fourth embodiment.

An architecture in FIG. 15A is that synaptic circuit blocks (D₁, . . . ,D₄) output the signals defined as results of implementing the timewindow integration using the weighting coefficient mapped to thesynaptic weight value with respect to the output pulses from theintegration layer neurons (N₁, . . . , N₄), and the feature detectionlayer neuron (M₁) adds the signals after being integrated, therebyforming an internal state of the neuron.

FIG. 15B shows a configuration of each of the synaptic circuits D_(j).The synaptic circuit D_(j) is structured to input a pulse signal fromthe anterior hierarchy, a weighting coefficient function signal and atiming signal for taking synchronization between the layers. A timewindow integration circuit 1501 executes the time window integration ofan input pulse signal and a weighting coefficient function. An outputsignal generation circuit 1502 generates and outputs a signalcorresponding to a result of the time window integration. Note that theoutput signal generation circuit 1502 may output directly the result ofthe time window integration.

A weighting coefficient value of the weighted time window integrationcarried out in each of the synaptic circuits is given as a function oftime as in the first embodiment. What is characteristic of the fourthembodiment is, however, that a peak value thereof corresponds to S_(ij)in the formula (4) in the embodiment 1. Note that a profile of theweighting coefficient function of the time window integration may be, asa matter of course, set in the way described above.

The peak value of this weighting coefficient function is defined as asynaptic connection weight value configuring the receptive fieldstructure needed for detecting that a local feature (which is, e.g., alocal feature as shown in FIG. 7C when the feature detection layerneuron detects a triangle) at a predetermined low-order level has aproper spatial geometrical relationship so as to shape the featuredetected by the feature detection layer neuron.

The weighting coefficient function can be set corresponding to any of acase where the output from the feature integration layer neuron is aphase modulation pulse and a case where the same output is a pulse widthmodulation signal. In the former case, however, a profile ofdistribution of this value is, unlike the profile shown in FIG. 7B,asymmetrical within each sub time window. In the latter case, theweighting coefficient function takes a fixed value corresponding to thesynaptic connection weight value without depending on the time.

For example, in the case of inputting the phase modulation pulse signal,the weighting coefficient function linearly decreases corresponding to adelay time from an arrival predetermined time if a detection level fordetecting the local feature concerned comes to its maximum. In the caseof the pulse phase modulation signal, there is multiplied the weightingcoefficient value of a level corresponding to a phase delay (which is adelay from the reference time, given by the timing signal, of the pulsearrival time) corresponding to a magnitude of the output of the featureintegration layer neuron. For instance, a substantial product-sumoperation of the synaptic weight value and the neuron output isperformed by decreasing the weighting coefficient value linearlycorresponding to the delay of the pulse signal arrival time.

Further, when the pulse width modulation signal corresponding to theneuron output is inputted to the synaptic circuit, a profile of theweighting coefficient function may be set so that a result of timeintegration of the time-varying weighting coefficient function and thepulse width modulation signal is mapped to a result of multiplication ofthe integration layer neuron output and the synaptic connection weightvalue.

When executing the time window integration, normally the featuredetection layer neuron, the feature integration layer neuron existingwithin the receptive field thereof and the synaptic circuit, input andoutput the pulse signals in synchronization with the predeterminedtiming signals. This timing signal may involve the use of the signalfrom the pacemaker neuron as in the first embodiment, or the clocksignal supplied from outside.

A waveform of the weighting coefficient signal may be supplied fromoutside by a function waveform generator, or alternatively a digitalcircuit generates a digital waveform by LUT (Look-Up Table) method and afunction generation method, and thereafter the digital waveform isconverted by a D/A converter into an analog waveform (Morie, et al.,“Study and Research for AD Merged System Image Feature Extraction LSIand Natural Image Recognition System Using the Same LSI”: Study ofSpecified Field in 2000, Mixed Integrated Systems for Real-TimeIntelligent Processing, pp. 41–47). Note that it is easy to generate ahigh-precision voltage waveform serving as a function of time.

As in the fourth embodiment, the product with the synaptic weight can becalculated by the time window integration with respect to the pulsemodulation output signal from the neuron, whereby the high-precisionproduct-sum operation can be actualized with the compact circuitarchitecture.

A weighting coefficient signal generation circuit 1601 distributes, asshown in FIG. 16, an arbitrary analog non-linear function (function oftime) supplied from outside or generated with a high precision withinthe chip as a weighting coefficient signal to each synaptic connectioncircuit in the synaptic circuit block to the feature detection layerneurons related to the same feature class.

After undergoing the time window integration in the synaptic circuitblock, the respective signals (pulse signals) to be outputted areoutputted in branch to the predetermined neuron circuits inside theneuron circuit block, wherein a summation is taken. Referring again toFIG. 16, each of the synaptic circuits arrayed line by line within thesynaptic circuit block supplies this weighting coefficient signal toevery synaptic connection circuit to the feature detection layer neuron,whereby an arbitrary synaptic connection weight distribution within onereceptive field can be generated as a spatiotemporal function.

FIG. 17 illustrates an outline of an architecture of an image inputsystem (for instance, a camera, a video camera, a scanner and so on)mounted with the pattern recognition system as an object detection(recognition) system, which is based on the parallel pulse signalprocess involving the elements in the architecture shown in FIGS. 15Aand 15B as basic elements.

Referring to FIG. 17, a photographic system 1701 includes an imagingoptical system 1702 containing a photographic lens and a drive controlmechanism for zoom photography, a CCD or CMOS image sensor 1703, animaging parameter measuring portion 1704, an image signal processingcircuit 1705, a storage portion 1706, a control signal generationportion 1707 for generating control signals for control of imagingconditions, a display 1708 serving as a viewfinder such as EVF etc, astroboscope light emitting portion 1709 and a storage medium 1710.Further, the photographic system 1701 further includes an objectdetection (recognition) system 1711 (the pattern recognition systemconstructed of the parallel pulse signal processing circuit having thehierarchical structure in the embodiments discussed above).

This object detection (recognition) system 1711 in this photographicsystem 1701 detects (an existing position and a size of), for example, aface image of a pre-registered figure from within a picturephotographed. When the position of this figure and a piece of size dataare inputted to the control signal generation portion 1707, the controlsignal generation portion 1707 generates, based on an output from theimaging parameter measuring portion 1704, control signals for optimallycontrolling a focus on this figure, exposure conditions, a white balanceand so on.

The pattern detection (recognition) system according to the presentinvention is thus utilized for the photographic system, as a result ofwhich the detection of the figure etc and the optimal photographiccontrol (AF, AE etc) based on this detection can be attained byactualizing the function of surely detecting (recognizing) the objectwith a low consumption of electricity and at a high speed (in realtime).

According to the embodiments discussed so far, the pulse signalprocessing circuit includes the modulation circuit for inputting theplurality of pulsed signals from the different arithmetic elements andmodulating in common the predetermined signals among the plurality ofpulsed signals, and the modulated pulse signals are outputted in branchto the different signal lines, wherein the modulation circuit, whichshould provide the predetermined modulation amount to the plurality ofpulsed signals, is not required to be set for every pulse (everyinput-side signal line) in the signal processing system for transmittingthe pulsed signals between the plurality of arithmetic elements in a waythat executes the predetermined modulation on the pulsed signals, andthe circuit scale can be downsized.

Although the present invention has been described in its preferred formwith a certain degree of particularity, many apparently widely differentembodiments of the invention can be made without departing from thespirit and the scope thereof. It is to be understood that the inventionis not limited to the specific embodiments thereof except as defined inthe appended claims.

1. A pulse signal processing circuit comprising: a plurality ofarithmetic elements for outputting pulse signals; and at least onemodulation circuit each for receiving a plurality of pulse signals fromdifferent arithmetic elements, modulating in common a plurality ofpredetermined pulse signals among the plurality of pulse signals, andoutputting the modulated pulse signals in branch to different signallines, respectively, wherein said at least one modulation circuit eachgives a time-differing modulation characteristic with respect to aprocess of the same input data.
 2. A pulse signal processing circuitaccording to claim 1, wherein said at least one modulation circuit eachis a delay circuit for giving a predetermined delay to the plurality ofpredetermined pulse signals among the plurality of pulse signals.
 3. Aparallel processing circuit comprising: a plurality of neuron elements;and synaptic connection elements for connecting said neuron elements,wherein said synaptic connection elements are constructed by use of saidpulse signal processing circuit claimed in claim
 1. 4. A parallelprocessing circuit comprising: a plurality of neuron elements; andsynaptic connection elements for connecting said neuron elements, whichcomprise, a plurality of arithmetic elements for outputting pulsesignals, and at least one modulation circuit each for receiving aplurality of pulse signals from different arithmetic elements,modulating in common a plurality of predetermined pulse signals amongthe plurality of pulse signals, and outputting the modulated pulsesignals in branch to different signal lines, respectively, wherein saidsynaptic connection element shares a plurality of synaptic connectionsas one circuit on the basis of a distribution symmetry of synapticconnections to said predetermined ones of said neuron elements.
 5. Apattern recognition system comprising: data input means for inputtingdata of a predetermined dimension; a plurality of data processingmodules having feature detection layers for detecting a plurality offeatures; and output means for outputting a result of a patternrecognition, wherein said data processing module includes a plurality ofarithmetic elements connected to each other by predetermined synapticconnection means, each of said arithmetic elements outputs a pulsesignal at a frequency or timing corresponding to an arrival time patternof a plurality of pulses within a predetermined time window, said outputmeans outputs, based on the outputs of said plurality of arithmeticelements, a result of detecting or recognizing a predetermined pattern,said synaptic connection means includes at least one modulation circuiteach for receiving the plurality of pulse signals from said differentarithmetic elements, effecting a predetermined common modulation on aplurality of predetermined pulse signals among the plurality of pulsesignals, and outputting in branch the modulated pulse signals todifferent signal lines, respectively, and said at least one modulationcircuit each gives a time-differing modulation characteristic withrespect to a process of the same input data.
 6. A pattern recognitionsystem according to claim 5, wherein said at least one modulationcircuit each is a delay circuit for giving a predetermined delay to theplurality of predetermined pulse signals among the plurality of pulsesignals.
 7. A pattern recognition system according to claim 5, whereinsaid at least one modulation circuit each is a pulse width modulationcircuit for effecting a predetermined pulse width modulation on theplurality of predetermined pulse signals among the plurality of pulsesignals.
 8. A pattern recognition system comprising: a modulationcircuit for receiving a plurality of pulse signals from differentarithmetic elements and giving a predetermined delay to each pulsesignal; and a branch circuit for outputting in branch the modulatedpulse signals in a predetermined sequence to different signal lines,respectively, in a way that gives a predetermined delay to each pulsesignal, wherein said modulation circuit gives a time-differingmodulation characteristic with respect to a process of the same inputdata.
 9. A pulse signal processing circuit comprising: a parallelmodulation circuit for receiving a plurality of pulse signals fromdifferent arithmetic elements and effecting a predetermined modulationin parallel on a plurality of predetermined pulse signals among theplurality of pulse signals; and integration means for integratingoutputs of said parallel modulation circuit, wherein said parallelmodulation circuit includes a plurality of time window integrationcircuits for effecting a predetermined weighted time window integrationwith respect to a plurality of predetermined pulse signals among themodulated pulse signals, each of said arithmetic elements output apredetermined pulse signal on the basis of a signal from saidintegration means, and said time window integration circuits multiplythe pulse signals by a predetermined weighting coefficient set for everypulse signal inputted in parallel to said parallel modulation circuitand configure receptive fields of said arithmetic elements.
 10. A pulsesignal processing circuit according to claim 9, wherein said parallelmodulation circuit is a pulse width modulation circuit for effecting apredetermined pulse width modulation on the plurality of predeterminedpulse signals among the plurality of pulse signals.
 11. A pulse signalprocessing circuit according to claim 9, wherein outputs of saidarithmetic elements are, after being stored in a predetermined memory,outputted in branch to other arithmetic elements.
 12. A pulse signalprocessing circuit according to claim 9, further comprising a weightingcoefficient signal generation circuit, wherein said time windowintegration circuits multiply the input pulse signals by a weightingcoefficient signal transmitted from said weighting coefficient signalgeneration circuit.
 13. An image input system executing a predeterminedimage input by use of said pulse signal processing circuit claimed inclaim 9.